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DETAILED ACTION 



Claim Rejections - 35 USC § 101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 



1. Claims 32 and 64 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. The preamble of the claims states 
"a machine readable medium storing machine-readable instructions..." Under the 
current guidelines used by the USPTO a machine readable medium is considered non- 
statutory. A computer-readable medium storing computer-readable instructions is 
currently considered statutory. 



Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed 
publication in this or a foreign country, before the invention thereof by the applicant for a patent. 



(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21 (2) 
of such treaty in the English language. 
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2. Claims 1, 2, 4-12, 14, 15, 19, 20, and 22- 31 are rejected under 35 U.S.C. 102(e) 
as being anticipated by Pryor et al. (USPN: 7042440), hereinafter Pryor. 

Regarding claims 1 and 31 , the claims are drawn to a method of operation and 
associated input device, Pryor discloses a computer input system that acquires images 
of an interactive space from multiple fields of view and then detecting an input target in 
the acquired images (Fig. 7, element Step 1) and then computes the coordinates of the 
detected target (Fig. 7, element Step 4). Pryor then calculates patterns of location, 
orientation and timing of the movement of the input target (Fig. 7, elements 7A), 
processes the information to determine if the calculated movement equals a target path 
to determine the movement to determine a command (Fig. 7, elements Step 7C and 8) 
and then executes the input instruction based on the gesture made by the input target 
(col. 43, lines 43-63). 

Regarding claim 2, Pryor discloses using stereoscopic pairs of images (Fig. 1c; 
col. 9, lines 15-35). 

Regarding claims 4 and 5, Pryor discloses using a threshold image to determine 
input location and calculating the centroid of a group of pixels (Fig. 1d; col. 12, lines 41- 
64). 

Regarding claim 6, Pryor discloses subtracting background pixels from 
foreground pixels in the received images (col. 12, lines 27-30). 

Regarding claim 7, Pryor discloses using cameras as the input devices to receive 
different images and then using the different images to determine the coordinates of the 
input target (col. 10, lines 19-49). The Examiner notes that all images received by a 
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camera appear as a two-dimensional image when the information is read from the 
camera. 

Regarding claims 8 and 9, Pryor discloses computing three-dimensional 
coordinates based on sets if two-dimensional data and calibration data for the cameras 
(col. 14, line 46 - col. 15, line 3). 

Regarding claim 10, Pryor discloses using input gestures based on input 
coordinates to manipulate a computer system (col. 27, line 43 - col. 28, line 43). This 
inherently requires associating the measured input locations with the times the locations 
occurred. Without such an association it would be impossible to determine if a gesture 
has occurred. 

Regarding claims 1 1 and 12, Pryor discloses using colors and associating them 
with the input target for determining the input made by the input target (Fig. 1e; col. 13, 
lines 19-57). 

Regarding claims 14 and 15, Pryor discloses determining trace routes of 
connected input targets in time and based on state changes of the input target (Fig. 7, 
element Step 7A). 

Regarding claims 19 and 20, Pryor discloses comparing an identified trace or 
gesture by a user to a predetermined representation of an input gesture (col. 27, line 43 
- col. 28, line 8). 

Regarding claims 22-29, Pryor discloses determining gestures and input motions 
and then manipulating items within a virtual environment of a game or otherwise. Pryor 
further includes the gestures to be associated with predetermined computer instructions 
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and manipulates objects in the virtual environment based on the gesturejnputs (col. 27, 
line 43 - col. 28, line 43). 

Regarding claim 30, Pryor discloses calculating input coordinates based on a 
pair of input cameras (Fig. 1c, elements 60 and 61 ). The actual position is determined 
by interpolating based on the input images to generate a view of the object within the 
interactive space. 

3. Claims 33, 34, 37, 39, 40, and 63 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Pulli et al. (USPN: 6771294), hereinafter Pulli. 

Regarding claims 33 and 63, the Examiner notes these claims are a method of 
operation and associated input device. Pulli discloses an input device and associated 
method of operation that includes displaying an image at a display location between a 
viewing space and interactive space (Figs. 2 and 3), acquiring images of the interactive 
space from at least one field of view (Fig. 2a, elements 100A and 100B are cameras), 
detecting an input target in the images, computing coordinates, identifying an instruction 
based on input coordinates and executing an instruction based on the coordinates (col. 

4, lines 15- 45 describe the input device; col. 7, line 17 - col. 8, line 2 describe one 
method of the user interacting with the device). 

Regarding claim 34, Pulli discloses using a portable electronic device (abstract). 

Regarding claim 37, Pulli discloses using a field of view disposed between the 
display location and the interactive location (Fig. 2a, the cameras on the lenses are 
between the display location (the lenses) and the interactive area (in front of the user). 
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Regarding claim 39, Pulli discloses using multiple fields of view (Fig. 2a, 
elements 100a and 100b are cameras). 

Regarding claim 40, Pulli discloses changing displaying information based on the 
information from comparing the input of the cameras. This would require interpolating 
position based on the information from both cameras and producing the images based 
on that interpolated input (col. 4, lines 15-45). 

4. Claims 33 and 35 are rejected under 35 U.S.C. 102(a) as being anticipated by 
Kurtenbach et al. (USPgPub: 2003/0142067), hereinafter Kurtenbach. 

Regarding claim 33, Kurtenbach discloses a display and interaction system that 
displays an image in a display location between a viewing space and an interactive 
space (Fig. 2, image space is in element 34, the interactive space is element 32), 
acquiring images of the interactive space from at least one field of view, detecting an 
input target in the images, computing coordinates and identifying and executing 
instructions based on the input (paragraph 24). The users hand is detected by cameras 
and then used to interact with the display. 

Regarding claim 35, Kurtenbach discloses the display location corresponding to 
an area in a desktop surface (Fig. 2, element 32). 
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Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pryor. 
Regarding claim 13, Pryor discloses all of the limitations except, "wherein the 

spatiotemporal input data structure is constructed in the form of a linked list of data 
records." 

The Examiner notes that the use of a linked list data structure to store input data 
would be a design choice for one skilled in the art. One skilled in the art would be 
motivated to utilize a linked list data structure because of the ability of a linked list to 
store a sequence of information in order with each piece of data being linked directly to 
the following (and sometimes preceding) data. It would be an obvious choice for one 
skilled in the art to use a linked list to store a sequence of input positions associated in a 
sequence of time. 

6. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pryor in 
view of Kumar et al. (USPN: 6204852), hereinafter Kumar. 

Regarding claim 3, as discussed above, Pryor discloses all of the information 
except, "wherein images of the interactive space are acquired from at least three 
different fields of view." 
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Kumar discloses a three-dimensional input system using three cameras (Fig. 1, 
elements 16, 18, and 19). 

At the time of invention it would have been obvious to one skilled in the art to 
combine the teachings of Pryor and Kumar. The motivation for combining the two 
references would be to use a different number of cameras or different camera set-up for 
determining the location of an input object within a three-dimensional input space. It 
would have been obvious that the use of a pair of binocular cameras or 3 cameras, or 
any other number of cameras would be a matter of design choice for determining input 
locations of an object within three dimensions. Therefore, it would have been obvious 
to combine the teachings of Pryor and Kumar to produce the method as described in 
claim 3. 

7. Claims 16- 18, and 21 are rejected under 35 U.S. C. 103(a) as being 
unpatentable over Pryor in view of Schmalstieg et al. (USPN: 6842175), hereinafter 
Schmalstieg. 

Regarding claim 16, as discussed above Pryor discloses all of the limitations 
except, "wherein identifying traces comprises computing coordinates of bounding 
regions encompassing respective traces." 

Schmalstieg discloses using a bounding box around a stroke to determine the 
location and coordinates of an input stroke (col. 21, line 61 - col. 22, line 14). 

At the time of invention it would have been obvious to one skilled in the art to 
combine the teachings of Pryor and Schmalstieg. Schmalstieg discloses methods of 
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determining the locations of strokes detected by cameras for an input system. This 
method includes using a bounding box to determine the stroke location. Although, 
Schmalstieg deals with 2-dimensional input the technique could be applied to each of 
the cameras used in the Pryor system to produce the same results. The motivation for 
combining the teachings of these references would be to apply known techniques of 
input trance determination as discussed by Schmalstieg to a three-dimensonal input 
system used to determine input traces and gestures of Pryor. At the time of invention it 
would have been obvious to one skilled in the art that the bounding box principles 
described by Schmalstieg could modified to be used with the camera input system of 
Pryor to determine the locations of input strokes in three-dimensional space. Thus, the 
combination of Pryor and Schmalstieg disclose the invention of claim 16. 

Regarding claims 17, Schmalstieg discloses the bounding box being part of a 
two-dimensional input gesture form a single camera (col. 21, line 61- col. 22, line 14). 

Regarding claim 18, the Examiner notes that it would be logically obvious to one 
skilled in the art that multiple bounding boxes from different camera angles could be 
used to determine a bounding region around a gesture made in three-dimensional 
space/ Logically, by defining bounding areas based on images from each camera, the 
three-dimensional region defined by the intersection of the bounding regions would 
therefore be a bounding region in three-dimensions for the detected input traces and 
gestures. 
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Regarding claim 21, Schmalstieg discloses using gestures to write letters and 
numbers using input gestures measured by the camera input system (col. 20, line 26 - 
col. 21, line 6). 

Claims 36 and 38 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Pulli. 

Regarding claim 36, the Examiner takes Official Notice that it is well-known in the 
art to produce wearable display glasses that utilize projection means to project images 
onto surfaces within the lenses of the glasses. The use of projection based systems 
mounted on the glasses are well-known and common in the area of head mounted 
displays. One skilled in the art would be motivated to use projection displays or other 
types of displays based on design choice and the type of glasses being used. 

Regarding claim 38, the Examiner takes Official Notice that it is well known in the 
art that a wearable computer system with display glasses and a camera integrated with 
the glasses can position the glasses in the middle of the field of view where it would 
intersect the central area of the display location. The position of the input cameras may 
be over the eyes, along the outsides of the frames or in a central position between the 
frames along the central optical axis. One skilled in the art would be motivated to 
position the input cameras in such a manner as to detect desired information for use 
with the input system whether the camera would be centrally located or located at other 
positions away from the displays. 
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8. Claims 41 -48, 52-60, and 62 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Pulli in view of Pryor. 

Regarding claim 41, as discussed above, Pulli discloses all of the limitations 
except, "computing calibration parameters for the multiple fields of view." 

Pryor discloses calculating calibration data for different cameras to correct for 
distortion of the images (col. 15, lines 1-3). 

At the time of invention it would have been obvious to one skilled in the art to 
combine the teachings of Pulli and Pryor. As both systems are drawn towards input of 
coordinates and locations based on camera images of an input device, it would have 
been logical that any of the camera and coordinate calculation methods described by 
Pryor could be used on the cameras and information used in the Pulli system. The 
cameras of the Pulli system are only different from the cameras of the Pryor system 
because the Pulli system is a wearable system and the Pryor system is a non-wearable 
system. The motivation would be to apply the calibration and calculation techniques for 
determining input coordinates disclosed by Pryor with the camera input system utilized 
by Pulli. Therefore, it would have been obvious to combine the teachings of Pulli and 
Pryor to produce the device of claim 41 . 

Regarding claim 42, Pryor discloses using the two-dimensional information from 
input images and calibration data to determine coordinates of the input target (col. 14, 
line 46- col. 15, line 3). 
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Regarding claims 43 and 44, Pryor discloses using a threshold image to 
determine input location and calculating the centroid of a group of pixels (Fig. 1d; col. 
12, lines 41-64). 

Regarding claim 45, Pryor discloses subtracting background pixels from 
foreground pixels in the received images (col. 12, lines 27-30). 

Regarding claim 46, Pryor discloses using cameras as the input devices to 
receive different images and then using the different images to determine the 
coordinates of the input target (col. 10, lines 19-49). The Examiner notes that all 
images received by a camera appear as a two-dimensional image when the information 
is read from the camera. 

Regarding claims 47 and 48, Pryor discloses determining trace routes of 
connected input targets in time and based on state changes of the input target (Fig. 7, 
element Step 7A). 

Regarding claims 52 and 53, Pryor discloses comparing an identified trace or 
gesture by a user to a predetermined representation of an input gesture (col. 27, line 43 
- col. 28, line 8). 

Regarding claims 55 - 62, Pryor discloses determining gestures and input 
motions and then manipulating items within a virtual environment of a game or 
otherwise. Pryor further includes the gestures to be associated with predetermined 
computer instructions and manipulates objects in the virtual environment based on the 
gesture inputs (col. 27, line 43 - col. 28, line 43). 
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9. Claims 49-51 and 54 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Pulli in view of Pryor and in further view of Schmalsteig. 

Regarding claim 49, as discussed above, the combination of Pulli and Pryor 
disclose all of the information except, "wherein identifying traces comprises computing 
coordinates of bounding regions encompassing respective traces." 

Schmalstieg discloses using a bounding box around a stroke to determine the 
location and coordinates of an input stroke (col. 21, line 61 - col. 22, line 14). 

At the time of invention it would have been obvious to one skilled in the art to 
combine the teachings of Pulli, Pryor and Schmalstieg. Schmalstieg discloses methods 
of determining the locations of strokes detected by cameras for an input system. This 
method includes using a bounding box to determine the stroke location. Although, 
Schmalstieg deals with 2-dimensional input the technique could be applied to each of 
the cameras used in the Pulli and Pryor systems to produce the same results. The 
motivation for combining the teachings of these references would be to apply known 
techniques of input trance determination as discussed by Schmalstieg to a three- 
dimensonal input system used to determine input traces and gestures of Pryor. At the 
time of invention it would have been obvious to one skilled in the art that the bounding 
box principles described by Schmalstieg could modified to be used with the camera 
input system of Pryor and Pulli to determine the locations of input strokes in three- 
dimensional space. Thus, the combination of Pulli, Pryor, and Schmalstieg disclose the 
invention of claim 49. 
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Regarding claims 50, Schmalstieg discloses the bounding box being part of a 
two-dimensional input gesture form a single camera (col. 21, line 61- col. 22, line 14). 

Regarding claim 51, the Examiner notes that it would be logically obvious to one 
skilled in the art that multiple bounding boxes from different camera angles could be 
used to determine a bounding region around a gesture made in three-dimensional 
space. 

Regarding claim 54, Schmalstieg discloses using gestures to write letters and 
numbers using input gestures measured by the camera input system (col. 20, line 26 - 
col. 21, line 6). 

Conclusion 

10. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Wells et al. (USPN: 5334991) discloses a head mounted display 
system using a projection method to display images on the glasses lens. Mann (USPN: 
6307526) discloses a head mounted display system with a centrally located camera. 

1 1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Steven E. Holton whose telephone number is (571) 272- 
7903. The examiner can normally be reached on M-F 8:30-5. 

If attempts to' reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amr Awad can be reached on (571) 272-7764. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 

Patent Application Information Retrieval (PAIR) system. Status information for 

published applications may be obtained from either Private PAIR or Public PAIR. 

Status information for unpublished applications is available through Private PAIR only. 

For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 

you have questions on access to the Private PAIR system, contact the Electronic 

Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 

USPTO Customer Service Representative or access to the automated information 

system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Steven E. Holton 
Division 2629 
March 18, 2007 
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